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Summary 

Objectives The phrase 'Initial Clinical Experience' in a manuscript's 
title implies that the described technique offers promise of future clinical 
relevance. We assessed, using rates of subsequent citations, the actual 
academic relevance of such articles in comparison to articles not 
containing the phrase. 

Design We searched ISI database for articles that included the studied 
phrase in theirtitles between 1975 and 2009 and grouped the results by the 
related medical specialty. We excluded articles from journals with no 
available impact factor. For each identified article, we extracted number of 
included patients, citations/year, the average impact factor of the 
publishing journal over the last five years and the proportion of articles 
published in the same journal that garnered zero subsequent citations. 

Setting Retrospective review of a scientific database. 

Participants None 

Main outcome measures Citation rate 

Results Among a total of 982,127 articles published in 186 journals 
representing eight major publishing medical specialties, 531 (0.05%) were 
Initial Clinical Experience articles. Thirty percent of Initial Clinical 
Experience articles were never cited compared with 7% of the 
overall article volume (p < 0.0001 ). Citations/year for Initial Clinical 
Experience articles were significantly lower than the median impact factor 
(p < 0.0001). There was no correlation between citations and number of 
patients described in the Initial Clinical Experience articles (p= 0.61). 

Conclusions Initial Clinical Experience articles are cited less 
frequently than the average, especially in Cardiovascular, Radiology and 
Ophthalmology journals. 



Introduction 

Authors of clinical studies frequently include 
modifiers to titles of articles, especially those 



focused on relatively new methodologies. These 
modifiers include words such as 'preliminary', 
'pilot', 'early' or 'initial'. It is not entirely clear 
whether authors are trying to impart the sheen 
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of novelty to their work or are simply trying to 
lower the expectations of reviewers and editors, 
so that the study can be given the 'benefit of the 
doubt' given its (apparent) newness. 

Editors typically strive to maximize the 
academic relevance of their respective journals. 
One of the most widely applied metrics of such 
relevance is the rate of citation of papers in a 
given journal. Although many measures of rela- 
tive rates of citation are available, journal impact 
factor represents the most widely applied metric. 
Impact factor simply counts the number of current 
year citations to the source items published in that 
journal during the previous two years. 1 

Publication of nascent technologies or method- 
ologies introduces both potential risk and reward 
for journal editors focused on maximizing 
high impact factors. If work that is described by 
the authors as 'early' subsequently enjoys wide- 
spread application, then publication of that early 
work likely will lead to high rates of citation and 
thus improves a journal's impact factor. However, 
if such early work fails to stimulate future studies, 
the journal's impact factor will suffer. As a con- 
tinuation of a previous effort of studying 'pre- 
liminary' and 'pilot' articles, 2 the purpose of the 
current study was to characterize the subsequent 
academic relevance of papers with titles contain- 
ing the phrase 'Initial Clinical Experience'. 

Methods 

Data selection 

In October 2011, we collected our study cohort from 
ISI Web of Knowledge database. 3 We searched 
between 1975, the earliest date provided by ISI, 
and 2009, to allow a minimum of two years for 
articles to be cited. We used the search phrase 
'Initial Clinical Experience' in the 'Title' search bar 
to identify articles of interest and, accordingly, the 
publishing journal. Articles from journals that had 
no reported impact factor on ISI were excluded 
from our studied population. The ISI database pro- 
vides not only the number of citations of the studied 
article and the impact factor of the publishing 
journal in the last five years, but also categorizes 
articles by medical specialty. Articles were categor- 
ized into one of the following groups: Anesthe- 
siology, Cardiovascular system, Neurosciences, 
Oncology, Ophthalmology, Radiology, Surgery, 



Urology and other specialties with less frequent 
Initial Clinical Experience publications (i.e. Gastro- 
enterology, Hepatology, Dermatology). We then 
extracted the number of citations of each included 
article, the publishing year, the five most recent 
impact factors of the publishing journals and the 
number of patients of each study. The five most 
recent values of each journal's impact factor were 
averaged. We also examined all articles that were 
published by the included journals over the same 
study period (1975-2009) and calculated the per- 
centage that were never cited in each group. 

Given the cumulative nature of number of cita- 
tions, older articles are more likely to have more 
citations than newer articles. To normalize older 
and newer articles, we calculated the 'citations 
per year' for each article as the number of citations 
garnered divided by the number of years since 
publication. 

Statistical analysis 

All statistical analyses were performed using the 
software packages Microsoft Excel 97 (Microsoft 
Corporation, Redmond, WA, USA) and JMP (SAS 
Institute, Cary NC, USA). Statistical significance 
was assigned to P values of <0.05. Continuous 
variables were presented as median with inter- 
quartile ranges due to deviations from normal dis- 
tributions as calculated using the Shapiro-Wilk 
test. Correlation between number of patients and 
citations per year was determined using linear 
regression. Two-tailed Fisher's exact test was used 
to analyse differences in the never-cited proportion 
among specialties. Wilcoxon rank-sum test was 
used to compare median citations per year with 
the average impact factor and to compare the 
median number of patients among specialties. 

Results 

Our search identified 581 articles from 201 jour- 
nals that contained the phrase Initial Clinical 
Experience in their titles. In total, 50 (9%) of 
these articles from 15 journals were excluded as 
the journals did not report an impact factor. As 
such, 531 articles from 186 different journals 
were included in our study. 

Table 1 summarizes the demographics of 
the studied articles and their publishing journals. 
Cardiovascular (33%, 174/531) and Radiology 
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Table 1 > 

Journal and article demographics 



Journal 
specialty 


Number of 
journals 


Total 
number 
of articles 


Initial Clinical Experience articles 

Number Median Median 
number of publication 
patients (IQR)* year 


All 


186 


982,127 


531 


19 (9-39) 


2001 


Anesthesiology 


7 


36,906 


12 (2.3%) 


12 (7-51) 


1996 


Cardiovascular 


37 


175,163 


174 (32.8%) 


20 (9-47) 


2000 


Neurosciences 


19 


90,699 


40 (7.5%) 


15 (9-46) 


2004 


Oncology 


15 


85,131 


36 (6.8%) 


10 (6-28) 


2005 


Ophthalmology 


7 


55,204 


19 (3.6%) 


17 (13-43) 


1998 


Other 


57 


318,398 


60 (11.3%) 


14 (7-23) 


1998 


Radiology 


20 


100,037 


112 (21.1%) 


25 (13-40) 


2000 


Surgery 


16 


60,107 


28 (5.3%) 


25 (11-39) 


2003 


Urology 


8 


60,482 


50 (9.4%) 


18 (6-55) 


2006 



'Interquartile range 



/ Table2 

Comparison of citation rates between Initial Clinical Experience articles and overall journal publications 

Journal Articles with zero citations Median citations per year 



specialty 



All articles 



Initial Clinical 
Experience 
articles 



p value 



All articles 
( journal impact 
factor) (IQR)* 



Initial Clinical 

Experience 
articles^ (IQR)" 



p value 



All 

Anesthesiology 

Cardiovascular 

Neurosciences 

Oncology 

Ophthalmology 

Other 

Radiology 

Surgery 

Urology 



68,539 (7%) 
2327 (6%) 

14,250 (8%) 
4608 (5%) 
2350 (3%) 
1475 (3%) 
31,363 (10%) 
5043 (5%) 
3584 (6%) 
3539 (6%) 



159 (30%) 
2 (17%) 
62 (36%) 
11 (28%) 
11 (31%) 
9 (47%) 
13 (22%) 
27 (24%) 
4 (14%) 
20 (40%) 



<0.0001 

0.17 
<0.0001 
<0.0001 
<0.0001 
<0.0001 

0.0072 
<0.0001 

0.08 
<0.0001 



3.0 (1 
2.5 (0 
2.9 (1 
2.7 (1 
4.4 (3 
2.9 (2 
1.9 (0 
3.0 (2 
1.7 (1 
2.2 (2 



8-4.5) 

8- 3.2) 
4-4.2) 

9- 3.0) 
6-4.4) 
1-3.2) 
9-2.6) 

1- 5.7) 

2- 2.6) 
1-4.0) 



1.5 (0.5-3.9) 

1.2 (0.3-2.7) 

1.3 (0.3-3.0) 

2.4 (1.0-5.5) 
2.3 (0.8-7.5) 
1.7 (0.8-4.0) 
1.0 (0.4-2.5) 

2.2 (0.8-6.2) 

1.3 (0.2-4.0) 

1.6 (0.4-4.8) 



<0.0001 

0.20 
<0.0001 

0.56 

0.21 

0.17 

0.0372 

0.053 

0.46 

0.24 



'Interquartile range 
tExcluding zero citation articles 



(21%, 112/531) represented the most frequently 
publishing specialties of initial clinical experience 
articles. Median number of included patients in 
Initial Clinical Experience articles was found to 
be higher in Radiology, Surgery and Cardio- 
vascular journals without significant differences 
among specialties (p = 0.25). There was no corre- 
lation between the median number of patients in 
each specialty and the subsequent citation rate 
(p = 0.61). 



Table 2 summarizes the citation rates of Initial 
Clinical Experience articles compared with the 
overall published volume. Overall, 159 (30%) of 
531 Initial Clinical Experience articles were never 
cited. The proportion of never-cited articles was 
significantly different among journal specialties 
(p = 0.0420). Journals with the highest proportion 
of never-cited articles were specialized in Oph- 
thalmology (47%, 9/19), Urology (40%, 20/50) 
and Cardiovascular (36%, 62/174). A significantly 
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higher proportion of Initial Clinical Experience 
articles were never cited (30%, 159/531) compared 
with all articles (7%, 68,559/982,127) (p < 0.0001). 

Of the Initial Clinical Experience articles that 
were cited, the median citations per year was not 
significantly different among journal specialties 
(p = 0.127). Neuroscience, Oncology and Radi- 
ology articles had the highest median citations 
per year. Cardiovascular and Other articles were 
found to have significantly fewer citations per 
year than the average impact factor of their pub- 
lishing journals (p < 0.0001, p = 0.0372 respect- 
ively). Overall, Initial Clinical Experience articles 
had significantly lower rates of citations per year 
in comparison to the average impact factor of the 
publishing journals (p < 0.0001). 

Discussion 

Our findings suggest that articles with the phrase 
Initial Clinical Experience in their title are less 
likely to be cited than other articles published in 
the same journals. This indicates that the volume 
of subsequent dependent research work does not 
reach the expected degree that is promised from 
the title. Thus, the lower average of citations 
would likely have a negative effect on the 
publishing journal's impact factor. Our results 
are comparable to an earlier study showing that 
only 27% of 'preliminary' or 'pilot' articles were 
subsequently followed by a more definitive 
publication. 2 

Several studies have examined the impact of 
an article's title on subsequent citations. Howard 
et al. 4 suggested that methodological and review 
articles tend to be cited more frequently 
than other article categories in Drug and Alcohol 
journals. Montori et al. 5 and Bhandari et al. 6 separ- 
ately further specified that systematic reviews are 
more cited than narrative ones. Jacques et al. 7 
concluded that number of citations was positively 
correlated with the length of the title and the 
presence of specific words like 'colon' or an 
acronym. They, however, found that reference to 
a specific country makes the article poorly cited. 



Our study of Initial Clinical Experience articles 
expands upon the effect of an article's category 
on its chances of success. 

The current study has several limitations. Due 
to conducting an automated search, we may 
have missed related articles that do not literally 
mention our search phrase. We also calculated 
the average impact factor of each journal over the 
last five years even if the article was published 
earlier. Finally, we were forced to exclude (9%) 
of initially identified Initial Clinical Experience 
articles that were published in journals that did 
not list impact factors. 

Conclusions 

Articles with the phrase Initial Clinical Experience 
in their titles are cited less frequently than average 
articles published in the same journals, especially 
in Cardiovascular, Radiology, and Ophthalmology 
journals. These articles likely negatively affect the 
impact factor of the publishing journal. 
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